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Program Evaluation for the Practitioner 

Using EVALUATION as a SCHOOL IMPROVEMENT Strategy 



D ata-driven decision making. This phrase is familiar to 
most schools and districts engaged in comprehensive school 



reform and improvement. It reminds practitioners that 
their plans have a greater likelihood of succeeding if 
the goals and strategies within them are based on solid 
information and not on hunches or habit. But where 
can schools get the data they need? As they strive for 
continuous improvement, how do they know which 
goals and strategies to keep or expand and which to 
drop? This month’s newsletter explains how schools 
can use program evaluation strategies to gather and 
analyze data and make informed decisions that 
contribute to continuous improvement. 
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What Is Program 
Evaluation? 

Quite simply, evaluation means taking a 
closer look at and getting feedback about an 
undertaking, with an eye to making a decision 
about its value. In schools, program evaluation 
means examining initiatives the school has 
undertaken — whether the initiative is an approach 
to literacy instruction or a program to support 
struggling students — to answer the question, 

"Is what we are doing working?" In school 
environments, evaluators are often seen as 
outsiders trying to get a better understanding of 
how well students are learning and teachers are 
teaching (Angelo & Cross, 1993). But evaluation is 
not just for outsiders. Many evaluation techniques 
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are easy to execute; can make use of data that are already being gathered; and can be performed 
on a scale that is practical for teachers, principals, and other school leaders. These "internal" 
evaluations can provide useful information about what is happening in the school and a strong, 
data-driven foundation for designing, implementing, and improving strategies that promote student 
achievement. 

Why Is Program Evaluation Important 
in Comprehensive School Reform and 
Improvement? 

Most school improvement plans contain multiple goals and dozens of strategies that the staff will 
implement in support of increased student achievement. Fewer plans, though, identify how the staff 
will determine if those strategies have been implemented as intended or have produced desired 
outcomes. This is where program evaluation proves useful by providing the feedback practitioners 
need to make those determinations. The results obtained from an evaluation can suggest ways to 
modify the implementation of a practice or uncover a need for more professional development to 
support its implementation. With programs that take longer to provide significant improvements, 
informal evaluation strategies can track initial changes in outcomes. Early feedback, especially 
indications of initial success, can help build support for a program and provide an early warning of 
potential problems so they can be addressed (Hansel & Cavell, n.d.). 

How Is It Done? 

Even when practitioners understand why program evaluation is important, they often do not 
feel they have the skills to do it. Although a complex or large-scale evaluation is best left to the 
professional evaluator, the following five steps can be used by practitioners in any setting to 
effectively gather and interpret useful data. 

step i. Set the stage and clarify goals. 

Often, the best time to make decisions about 
evaluation is when a program is being designed. 

i-writing session can 

provide an opportunity to discuss the proposed 
program, why it is being initiated, and its 
intended effects. For example, a school staff 
might decide to start a voluntary afterschool 
tutoring program in mathematics and English. 

They would begin by discussing the target 
audience and how the program will operate, 
but they also would clarify what they hope to 
accomplish: improved passing rates, for instance, 
or a greater number of students proficient on 
state assessments. The clearer and more specific 
the goals for the program are from the outset, 
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the more likely it is that the evaluation will be 
able to determine if those goals were achieved. 
The participation of the staff in the initial design 
will also help to ensure their full and active 
participation in the evaluation activities. 

step 2 . Create evaluation questions. 

The designers of an informal evaluation can use 
their goals to create questions that the evaluation 
will answer. In the case of the tutoring program, 
the staff might decide that it wants to know the 
following: 

• Was the tutoring program implemented as 
designed? Did implementation support the 
established goals? 




• Did participation in the program have a positive 
effect on student achievement in mathematics 
and reading? 

• How many students chose to participate in the 
tutoring program? Did the number of participants 
grow over time? Did participation improve 
participants' outlook toward school? 

step 3. Decide what data are needed 
and how to collect them. 

Once the evaluation questions have been 
formulated, the next step is to decide what 
data are needed to answer them. Practitioners 
should avoid designing elaborate data collection 
methods and focus instead on using data that 
are already available or easy to collect. Using 
team planning time or a portion of every staff 
meeting to discuss and informally summarize 
tutoring sessions will provide implementation 
data that address the first question. If the school 
already uses pretests and posttests or other 
assessment measures periodically throughout 
the year, gathering student achievement data 
for the participants in the afterschool program 
will be relatively easy. Answering the student 
participation questions might require new data 
collection tools, but here, too, the emphasis 
should be on simplicity and practicality. The first 
step would be to establish a system to track 
attendance. A sign-in sheet or computer logon 
screen would work. Participant interviews can 
provide data about students' interest in the 
program, if and how the program has improved 
their attitude toward school, and even what 
changes they would recommend. 

step 4. Analyze and interpret data. 

By analyzing and interpreting the data collected, 
the staff can make an informal judgment about 
how well the goals of the tutoring program have 
been met. Notes and recorded comments on 
the structure of the tutoring sessions, lesson 
plans, and activities can be examined. Average 
gains in test scores can be computed and trend 



data examined. Percentages, averages, and 
other statistical measures can be calculated to 
determine patterns in daily attendance at tutoring 
sessions. Student interview data can be analyzed 
to determine why students participate and how 
the program can be improved. 

The timing of the data analysis and interpretation 
should be driven by the evaluation questions. 

Staff members might decide to interpret some 
data formatively — that is, while the program 
is being conducted — so they can see what 
implementation modifications and adjustments are 
suggested. Other data are best analyzed at the 
program's end (such as the end of the school year 
or semester) and used to assist in decision making 
about further intervention efforts. 

step 5. Use the results. 

After staff members have discussed the analyzed 
data from their afterschool program, they need 
to formulate one more important evaluation 
question: "What will we do with what we found 
out?" It is likely that the data collected provide 
useful information about what was accomplished 
but also highlight areas that need improvement. 
Armed with this information, the staff has what 
it needs to revise and adapt the program for the 
following year. 

Conclusion 

Evaluation strategies can and should be an 
essential component of school improvement 
planning for practitioners. The process provides 
staff with information about how well programs 
are implemented, what changes need to occur, 
and what difference, if any, the program is 
making. Evaluating eliminates guesswork and 
provides an objective framework forjudging the 
value of initiatives undertaken by a school. With 
the information gained from evaluating their 
programs, schools have data they can use to 
make informed decisions. 
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More information on program 
evaluation for practitioners can 
be found in Evaluating Whole- 
School Reform Efforts: A Guide for 
District and School Staff published 
in August 2000 by the Region 10 
Comprehensive Center and the 
Northwest Regional Educational 
Laboratory. The guide is available 
at www.nwrac.org/whole-school/ 
index.html. 
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